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5.2.3 Conditioning and Independence 

Here, we will discuss conditioning for continuous random variables. In particular, we will discuss 
the conditional PDF, conditional CDF, and conditional expectation. We have discussed conditional 
probability for discrete random variables before. The ideas behind conditional probability for 
continuous random variables are very similar to the discrete case. The difference lies in the fact that 
we need to work with probability density in the case of continuous random variables. Nevertheless, 
we would like to emphasize again that there is only one main formula regarding conditional 
probability which is 


P(A n B ) 

p (A I B) = , when P(B) > 0. 

Any other formula regarding conditional probability can be derived from the above formula. In fact, 
for some problems we only need to apply the above formula. You have already used this in Example 
5.17. As another example, if you have two random variables X and Y, you can write 

P(X G C,Y £D) 

P(X G C| Y G D) = £ D) -, where C,OcR. 

However, sometimes we need to use the concepts of conditional PDFs and CDFs. The formulas for 
conditional PDFs and CDFs of continuous random variables are very similar to those of discrete 
random variables. Since there are no new fundamental ideas in this section, we usually provide the 
main formulas and guidelines, and then work on examples. Specifically, we do not spend much time 
deriving formulas. Nevertheless, to give you the basic idea of how to derive these formulas, we start 
by deriving a formula for the conditional CDF and PDF of a random variable X given that 
X G / = [a, b]. Consider a continuous random variable X Suppose that we know that the event 
X G / = [a, b] has occurred. Call this event A. The conditional CDF ofXgivenH, denoted by 

F X\a( x ) orF X\a<X<b^’ is 


F xia (x) = P(X<x\A) 

= P(X< x\a < X< b) 
P(X<x,a<X< b ) 

= P(A) ' 

Now if x < a, then F x ^ A (x) = 0. On the other hand, if a < x < b, we have 
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*x\aV> 


P{X < x, a < X < b) 
P{A ]j 
P(a <X<x) 

P(A) 

f j&) ~ 

P/b) - F^a )' 


Finally, if x > b, then F x \ A (x) = 1. Thus, we obtain 


x > b 


F x\J< x ) 


F x( x ) ~ F x( a ) 
F x( b ) ~ F x( a ) 


a <x < b 


V 0 otherwise 

Note that since X is a continuous random variable, we do not need to be careful about end points, 
i.e., changing x > b to x > b does not make a difference in the above formula. To obtain the 
conditional PDF of X, denoted by f x \ A (x), we can differentiate F x \ A (x). We obtain 


fx\ a( x ) 


a < x < b 


otherwise 


It is insightful if we derive the above formula for 4 (x) directly from the definition of the PDF for 
continuous random variables. Recall that the PDF ofX can be defined as 


fx(x) = lim 
A—>0"* 


P(x<X<x + A) 


Now, the conditional PDF ofX given A, denoted by f x \ A (x), is 


fx\A^ x ) = lim 

A—>0 

= lim 


= lim 


P(x<X<x+ A\A) 


P(x <X<x + A,A) 

- A P(A) 

P(x < X < x + A, a < X < b) 

- A P(A) ' 
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The conditional expectation and variance are defined by replacing the PDF by conditional PDF in 
the definitions of expectation and variance. In general, for a random variable X and an event A, we 
have the following: 
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roo 

E[X\A\ = J _ o0 xf X \ A (x)dx, 

roo 

E[g{X)\A\ = J - <x g(x)f x]A (x)dx, 
Var(X\A) = E[X 2 \A] - {E[X\A]) 2 


Example 5.20 

Let X ~ Exponential{\). 

a. Find the conditional PDF and CDF of X given X > 1. 

b. Find E[X\X> 1], 

c. Find Var(X|X > 1). 

• Solution 


° a. Let A be the event that X > 1. Then 


Thus, 


roo 

P(A ) = j | e X dx 

1 

e 


fx |x>i( x ) 



X > 1 

otherwise 


For x > 1, we have 

Fjfd-FAl) 

F x\ a (A) = Td) 

= 1 -e~ x+1 . 

Thus, 


b. We have 


r\-e~ x+X 

F X\J< X ) = | 

lo 


x > 1 

otherwise 
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c. We have 


Thus, 


E[X\X> 1] = 


|*00 

J 1 X fx\x> 1 (X)dx 

xe~ x+l dx 

roo _ 

e\ | xe X dx 



2 


e~ 

e 


= 2 . 


E[X 2 \X>\] = j“x 2 / x|x>1 (x)dx 




x 2 e~ x+1 


dx 



o 

xe 


X dx 


= e[~ 2e~ x ~ 2xe~ x - x 2 e _x j” 

5 

= e~ 
e 

= 5. 


Var(X|X> 1) = E[X 2 \X> 1] - (E[X\X> l]) 2 
= 5-4 = 1. 


Conditioning by Another Random Variable: 


If X and Y are two jointly continuous random variables, and we obtain some information regarding 
Y, we should update the PDF and CDF of X based on the new information. In particular, if we get to 
observe the value of the random variable Y, then how do we need to update the PDF and CDF of Jf? 
Remember for the discrete case, the conditional PMF ofX given Y = y is given by 


p x\ r^/l-ly) 


p x/ x iXj) 


Now, if X and Y are jointly continuous, the conditional PDF ofXgiven Y is given by 
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fxfa>y) 

This means that if we get to observe Y = y, then we need to use the above conditional density for 
the random variable X. To get an intuition about the formula, note that by definition, for small A x 

and A we should have 

P(x<X<x + A x \y<Y<y + A,,) 

f X \y( x \y) ~ -7- (definition of PDF) 

A X 

P(x<X<x + A x ,y < Y<y + A } ) 

~ P(y <Y<y + A } )A X 
f xy (x,y)A x A y 
” /yOOA/. r 

/y(y) 

Similarly, we can write the conditional PDF of Y, given X = x, as 


For two jointly continuous random variables X and Y, we can define the 
following conditional concepts: 

1. The conditional PDF of X given Y = y: 

f x ^ x -y) 

fx ' 

2. The conditional probability thatX G A given Y = y: 

P{X G A | Y = y) = j/ xj y(x | y)dx 

3. The conditional CDF ofX given Y = y: 

F X \ y( x b) = P(X <x\Y = y) = \ X -J X \ y(x\ y)dx 
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Example 5.21 Let X and 7 be two jointly continuous random variables with joint PDF 


fxfa>y) = 


2 2 

x y xy 

— + — + — 
4 4 6 


0 <x < 1,0 <y < 2 


otherwise 


For 0 < y <2, find 

a. the conditional PDF ofX given 7 = y; 

b. P(X < j | Y = y). 

• Solution 

o a. Let us first find the marginal PDF of 7. We have 


2 2 

rix y xy 

/y0 ; ) = Jo7 + 7 + ^ dx 


3y 2 +y+ 1 


12 


for 0 < y < 2. 


Thus, for 0 < y < 2, we obtain 


fxy( x >y) 

f x\v( x \y) = 


3x 2 + 3 y 1 + 2 xy 
3y 2 +7+l 


for 0 < x < 1. 


Thus, for 0 < y < 2, we have 


fx | y( x It) 


3x 2 + 3y 2 + 2xy 
ly 2 +y+\ 


b. We have 


0 < x < 1 


otherwise 
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Conditional expectation and variance are similarly defined. Given Y = y, we need to replac efj^x) 
by f x | y (x \y) in the formulas for expectation: 



Example 5.22 

LetXand Y be as in Example 5.21. Find E[X\ Y = 1] and Var(X| Y = 1). 
• Solution 

o 
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So we have 


E[X\ Y~ 1] = J^x/^ y(x| 1 )dx 

, i 3x 2 + 3 y 2 + 2xy 
“J°' Y 3/+J+1 lj;=1 

»] 3x 2 + 3 + 2x 

= nX 7 —:-— dx 

J0 3 + 1 + 1 


dx 

(y = i) 


= ~| 0 3x 3 + 2x 2 + 3x dx 
7 

~~ 12 ’ 

E[X 2 1 7 = 1] = {“^x 2 /^ /pc | \)dx 

= ~| 0 3x 4 + 2x 3 + 3x 2 dx 

21 
“ 50' 


Var(X| 7=1) = E[X 2 \ Y = 1] - (E[X\ Y = l]) 2 


21 

50 

287 

3600 


7 

12 


Independent Random Variables: 


When two jointly continuous random variables are independent, we must have 


fx\ y( x \y) = /■*(*)• 


fxY( X ’>’) 

That is, knowing the value of Y does not change the PDF of X Since f X \ y(x |y) = ^ — , we 
conclude that for two independent continuous random variables we must have 


fx&X) =frfx)fy(y)- 


Two continuous random variables X and Y are independent if 
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f X y( x , y ) = f/ x )fy(y), for all x, y. 

Equivalently, X and Y are independent if 

F X y( x , y ) = F%( X )F y(y), for all x, y. 

IfX and Y are independent, we have 

E[XY] = EXEY, 

E[g(X)h(Y)] = E[g(X)]E[h(Y)]. 


Suppose that we are given the joint PDF fxyiy, y) of two random variables X and Y. If we can write 

fxY( x >y) = / i ( x )/ 2 ( f )> 


then X and Y are independent. 


Example 5.23 

Determine whether X and Y are independent: 


,2e~ x ~ 

2>; x,y>0 

a -fx^-y) = \ 


lo 

otherwise 

, 8xy 

0 <x <y < 1 

b -fxA x >y) = { 


VO 

otherwise 

• Solution 



a. We can write 


fx^ x X) = [e X u( x )\[2e 2y u(y)], 
where u{x) is the unit step function: 


H x> 1 

u ( x ) |q otherwise 

Thus, we conclude that X and Y are independent, 
b. For this case, it does not seem that we can write fxy(x,y) as a product of some 

/] (x) and Jy(y). Note that the given region 0 < x < y < 1 enforces that x < y. That 

is, we always have X < Y. Thus, we conclude that X and Y are not independent. 
To show this, we can obtain the marginal PDFs ofX and Y and show that 
f x ^ x X) tfjfaYJy), for some x,y. We have, for 0 < x < 1, 
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Jxydy 
= 4x(l -x 2 ). 


Thus, 


/ 4x(l — x 2 ) 0 < x < 1 

= | 

V 0 otherwise 


Similarly, we obtain 


,4>’ 3 0 < y < 1 

fy(y) — 

10 otherwise 


As we see, f XY (x, y) 7 ^ thus X and Y are NOT independent. 


Example 5.24 

Consider the unit disc 


D= {{x,y)\x 2 +y 2 <l). 

Suppose that we choose a point ( X, Y) uniformly at random in D. That is, the joint PDF of X and Y is 
given by 

,c (x,y) e D 

fxfc,y) = | 

v 0 otherwise 

a. Find the constant c. 

b. Find the marginal PDFs/j^x) and j\(y). 

c. Find the conditional PDF of X given Y = y, where -1 < y < 1. 

d. Are X and Y independent? 

• Solution 

o a. We have 


|* GO roo 

i = J -ooj -J x P^ dxd y 

= ff c dxdy 


D 


= c(area of D) 
= c(n). 
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Thus, c = 

n 

b. For -1 < x < 1, we have 


r oo 

M x ) = J -o/xAtMt 

= J-j-* 2 ; dy 


-V 

n ’ 


■x 2 . 


Thus, 


Similarly, 


c. We have 


; y/l - x 2 -1 < x < 1 


/xM = 


-vr 


'J 


/yO) = 


fxy( x ,y) 
fx ^ x]y) = ~m~ 


otherwise 


1 <y< 1 


otherwise 


2 a/ 1 “t 


y/l -y 2 <x < ^/l 


■r 


otherwise 


Note that the above equation indicates that, given 7 = y, X is uniformly 
distributed on [ - y]l ~y 2 , 1 ->’ 2 ]. We write 


X\Y = y ~ Uniform ( ~yj\~y 2 , -\J 1 ). 


d. Are X and 7 independent? No, because /^x, y) f ./^(xj/yfy). 
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Law of Total Probability: 

Now, we'll discuss the law of total probability for continuous random variables. This is completely 
analogous to the discrete case. In particular, the law of total probability, the law of total expectation 
(law of iterated expectations), and the law of total variance can be stated as follows: 

Law of Total Probability: 

P(A) = \\P(A \X=x)f x (x) dx (5.16) 

Law of Total Expectation: 

E[Y\ = \\E[Y\X=xYj^) ^ (5.17) 

= E[E[Y\X]] 

Law of Total Variance: 

Var(T) = L[Var(T|V)l + Xw(E[Y\X\) (5.18) 

Let's look at some examples. 

Example 5.25 

Let X and Tbe two independent Uniform (0, 1) random variables. Find P(X 3 + Y > 1). 

• Solution 

° Using the law of total probability (Equation 5.16), we can write 

P(X 3 + Y > 1) = [t y P(X 3 + Y > I | X=xVjfx) dx 
= j 0J P(x 3 + Y > 1 \X = x) dx 

= jo^T > 1 - x 3 ) dx (since X and Y are independent) 

= j 0 x 3 dx (since Y ~ Uniform(0, 1)) 

1 

~ 4' 
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Example 5.26 

Suppose X ~ Uniform^ 1,2) and given X = x, Y is an exponential random variable with parameter 
X = x, so we can write 


Y\X = x ~ Exponential(x). 


We sometimes write this as 


Y\X 


Exponential(X). 


a. Find EY. 

b. Find Var(Y). 

• Solution 

o a. We use the law of total expectation (Equation 5.17) to find EY. Remember that if 

1 


Y ~ Exponential ^), then EY = -. Thus we conclude 


E[Y \X = x] = 


Using the law of total expectation, we have 


EY = 


\™_n E [Y\X= x\f^x)dx 
= |j£’[T|X = x] ■ 1 dx 

= (, ~dx 
J1 x 

= ln2. 

Another way to write the above calculation is 

EY = £’[£'[T|X|] (law of total expectation) 


= E 


' 1 

X 


1 

(since E’fFIX] = —) 


\\\ dx 


x 

= ln2 


b. To find Var(Y), we can write 


https://www.probabilitycourse.com/chapter5/5_2_3_conditioningJndependence.php 


14/15 



9/17/2018 


Conditioning and Independence | Law of Total Probability 


Var(Y) = E [¥ 2 ] ~ (. E[Y \) 2 
= E[Y 2 ] ~ (ln2) 2 


= E[E[Y 2 \X\] -(ln2Y 


(law of total expectation) 


= E — - (ln2) 


(since Y\X ~ Exponential^ 0) 


(•2 2 , 

= jj — dx ~ (ln2) 

X 

= 1 - (ln2) 2 . 

Another way to find Var(Y) is to apply the law of total variance: 

Var(h) = £[Var(7|X)] + Var (E[Y\X\). 
Since Y\X ~ Exponential(X), we conclude 


Therefore 


E[Y\X] = 


Var(Y\X) = 


1 l l 

Var (Y)=E — + Var - 


E X +E X - (e\-]Y 


X 2 X 2 \ X 


<— previous 
next —> 


= E - - (ln2) 


= 1 - (ln2 V 
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